Sparse Relational Topic Models for Document Networks

Learning latent representations is playing a pivotal role in machine learning and many application areas. Previous work on the relational topic model (RTM) has shown promise on learning latent topical representations for describing relational document networks and predicting pairwise links. However under a probabilistic formulation with normalization constraints, RTM could be ineffective in controlling the sparsity of the topical representations, and often need to make strict mean-field assumptions for approximate inference. This paper presents the sparse relational topic model (SRTM) under a non-probabilistic formulation that could effectively control the sparsity via a sparsity-inducing regularizer and handle the imbalance issue in real networks via introducing various cost parameters for positive and negative links. The deterministic optimization problem of SRTM admits efficient coordinate descent algorithms. We also present a generalization to consider all pairwise topic interactions. Our empirical results on several real network datasets demonstrate better performance on link prediction and faster running time than the competitors under a probabilistic formulation.